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Introduction 


College graduates do not always use what they learn during school in their day-to-day work. In many 
cases, this disconnect between school and work is natural. Most jobs, even jobs closely related to a 
worker's field of study, include administrative and employer-specific tasks that are not taught in school. 
But sometimes workers do not use what they learn in school because they are employed far outside 
their field of study. Concerns about low levels of job relatedness have been particularly acute for 
science and engineering graduates. Research on job relatedness drives public debates about science and 
engineering worker shortages and occupational pathways. If science and engineering graduates are not 


using their education on the job, it could indicate a serious waste of human capital. 


The most common data source for studying job relatedness is the National Science Foundation’s 
National Survey of College Graduates (NSCG), which asks respondents about how their work on their 
“principal job” relates to their “highest degree.” The NSCG is a representative survey of all college 
degree holders in the US, with particularly robust data collection on science and engineering graduates. 
In recent years, research on the relatedness of workers’ jobs to their field of study has exploded. Much 
of this research focuses on the science and engineering workforce and has made use of the job 
relatedness data collected by the National Science Foundation. According to most studies, having a job 
unrelated to your field of study leads to job dissatisfaction and reduced earnings. Women in science and 
engineering are especially likely to exit jobs related to their field of study, though many disagree about 


what drives this. 


Existing research on job relatedness has not investigated what respondents think of when they 
report their job relatedness on the NSCG. What does a “related job” really mean to respondents, and 
what are we measuring and analyzing when we use these data? Research takes self-reports of job 
relatedness as given without detailing precisely what the NSCG measures. No work has been done to 
understand why respondents answer the job relatedness question in the way that they do. This 
uncertainty around the National Center for Science Engineering Statistics (NCSES) job relatedness 
questions differs from other subjective questions in the NSCG, like about job satisfaction, which are well 
understood and validated. A better understanding of the job relatedness question will improve research 


on job relatedness and will be valuable for future revisions of the NSCG survey instrument. 


To fill this knowledge gap, we analyze the NSCG to identify the individual and job characteristics 
that induce respondents to identify their job as related to their highest degree. Inferences about 


respondents’ interpretation of job relatedness questions from existing NSCG data are necessarily 


indirect. We address this weakness in the analysis of the NSCG data by conducting semistructured 
interviews with college graduates to directly investigate their understanding and interpretation of job 
relatedness. Both the NSCG data analysis and the semistructured interviews provide insights into the 


use of and possible revisions to the NSCG’s job relatedness variables. 


This report focuses on the job relatedness question included in the NSCG, reproduced below in box 
1. Respondents are asked about how their principal job relates to their highest degree, and they can 
answer that it is “closely related,” “somewhat related,” or “not related.” The 2015 NSCG (the most 
recent survey wave) suggests that over 55 percent of employed college graduates report working in 
closely related jobs, 25 percent report working in somewhat related jobs, and 20 percent are in jobs that 
are not related. These results have been stable over time, based on the results of past waves of the 


NSCG. Public-use NSCG microdata is available for survey waves going back to 1993. 


BOX 1 
NSCG Job Relatedness Question 


To what extent was your work on your principal job related to your highest degree? Was it: 
[Mark one answer] 

= Closely related 

= Somewhat related 


= Not related 


Note: See the 2015 NSCG questionnaire at https://www.nsf.gov/statistics/srvygrads/surveys/srvygrads-newrespond2015.pdf. 


The NSCG offers more reliable information on job relatedness than the available alternatives. Many 
surveys commonly used in social science research (e.g., the Census, the Current Population Survey, or 
the American Community Survey) include information on a respondent’s educational background and 
labor market experiences but do not collect information on how a respondents’ education connects to 
their work. Without an explicit question on job relatedness, such as the one asked in NSCG, this 
relationship can only be imperfectly inferred. Rothwell (2013), the Census Bureau,! Abel and Deitz 
(2015), Montt (2017), and Sellami et al. (2017) make independent, researcher-driven determinations 
about which occupations are related to a field of a study. By construction, these measures of job 


relatedness do not capture the diversity of job relatedness within an occupation-degree combination. 
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A related job could correspond directly with a field of study and a broad occupational category (e.g., 
an engineering major working as an engineer). Or, it could represent a worker’s subjective assessment 
of more detailed tasks. A manager of a production line in a manufacturing plant with an engineering 
degree could use her knowledge of engineering frequently and report that she is employed in a related 
job. Alternatively, a worker that reports being an engineer rather than a manager, but whose day is 
consumed by sales tasks, meetings, or supervisory responsibilities may, in frustration, report their job as 


unrelated to their degree. 


The analyses in this report distinguish between factors that cause a graduate to work in an 
unrelated job, such as family formation, lack of available related jobs, or weak job-finding networks, and 
factors that induce a graduate to identify a job as unrelated, such as job tasks or knowledge 
requirements. Though most existing literature discusses the former, this report reviews the latter. 
Several factors likely contribute to a survey respondent’s subjective assessment of their job as related 
to their field of study. The most important determinant of job relatedness is the respondent’s 
occupation, with some occupations hewing more closely to a field of study than other occupations. 
However, we hypothesize that occupations are complex constructs characterized by substantial within- 
occupation heterogeneity in workers’ experiences. The details of aworker’s tasks, supervisory 
responsibilities, and outputs— mediated by their career expectations—will determine their response to 
questions about how their job relates to their field of study. When a worker spends time in a job, their 
prior expectations about that job are either validated or invalidated, thus potentially changing how they 
think about their job. See Brunhaver and colleagues (2018) for a discussion of this process in 


engineering. 


This report is the first focused attempt to understand how respondents think about the job 
relatedness question in the NSCG. It begins with a review of two important literatures: studies on the 
costs of low job relatedness for workers and studies that develop the “task approach” to labor markets, 
which provides a theoretical basis for our qualitative and quantitative data analyses. After the literature 
review, we explore whether differences in job relatedness are explained by differences in knowledge 
requirements on the job. Though knowledge requirements for a respondent’s field of study are strongly 
associated with job relatedness, they do not explain all observed variation in job relatedness. We then 
analyze quantitative data from the NSCG and qualitative data collected during interviews to identify 
the factors contributing to workers’ assessments of job relatedness. In addition to knowledge 
requirements, job tasks, federal support for research, and membership in professional associations are 
closely associated with job relatedness. Management tasks, which are typically contrasted with 


research and development responsibilities, were positively related to job relatedness in both the NSCG 
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analysis and the interviews. This report concludes with a discussion of major themes and 


recommendations for future NSCG waves. 


Literature Review 


This section reviews two literatures relevant to this report: one on the consequences and causes of low 
job relatedness and another on the “task approach” to labor markets. The extant literature uses the 
NSCG job relatedness measure either as an independent variable determining wages (or some other 
outcome) or as a dependent variable determined by workers’ decisions or external pressures. Both 
applications would benefit from a stronger understanding of the job relatedness variable. We also 
review the task approach to labor markets, because understanding jobs as equilibrium bundles informs 


our subsequent data analysis and interviews 


The Causes and Effects of Low Job Relatedness 


The literature estimating the change in earnings associated with working in an unrelated job is one of 
the most active areas of study using the NSCG’s job relatedness measures. These studies traditionally 
find there is a large earnings decline associated with working in an unrelated job. Robst (2007a), one of 
the first papers in the literature, finds that working in an unrelated job comes with a 10 percent 
reduction in earnings for women and a 12 percent reduction for men, compared with workers in closely 
related jobs. These reductions are not constant across majors; liberal arts and humanities majors are 
more likely to work in related job, but the costs associated with low job relatedness for these graduates 
are not statistically different from zero. In contrast, natural science majors are more likely to work in 
jobs that are related to their field of study, but the cost of working in an unrelated job for natural science 
majors is greater. The social sciences present an intermediate case. These costs are also apparent when 
using a combined measure of overeducation (i.e., greater educational attainment than is required for the 
job) and mismatch (Robst 2008). Using subsequent years of National Center for Science Engineering 
Statistics data, Bender and Heywood (2009) both confirms Robst’s (2008, 2007a) finding that low job 
relatedness is associated with reduced earnings and finds that it is also associated with reduced job 
satisfaction. Low job relatedness can also result in earlier retirement (Bender and Heywood 2017), and 


its costs are higher for lower-income people.” 


Robst (2007b) further demonstrates that the costs of low job relatedness vary by reasons for 


mismatch: when workers are in unrelated jobs because of reduced demand, job amenities, or other 
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constraints, they experience greater earnings reductions. Hunt (2016), another recent and widely cited 
analysis, uses the 2003 and 2010 NSCG to show that dissatisfaction with pay and promotion 
opportunities especially explain women’s exits from science and engineering. This finding contrasts with 
prior research that finds family-related constraints as women’s primary reasons for exiting jobs related 
to their science and engineering degrees (Preston 1994). Robst (2007b) emphasizes that reasons for 
working in an unrelated job are often endogenous to the wage itself, raising important concerns about 


attributing causality to these estimates. 


Educational experiences of a major or degree are not homogenous: the same major can cover 
diverse coursework. Course variation within a major affects the costs of low job relatedness. Silos and 
Smith (2015) finds that switching occupations is associated with higher earnings for graduates who took 
diverse courses, presumably because broader educational portfolios allow workers to adapt to new 


occupational settings. 


Working in an unrelated job is closely associated with entrepreneurial activity and self-employment 
(Bender and Roche 2013; Sell 2013; Stenard and Sauermann 2016). Self-employed workers are more 
likely to take on various management and sales responsibilities not directly related to their field of 
study. But in other cases, dissatisfaction with working in an unrelated job may push workers into self- 
employment. In Europe, workers who transition to self-employment increase their job relatedness, 
relative to the job they held before self-employment (Albiol Sanchez, Diaz-Serrano, and Teruel 2015). 
Among self-employed workers, those who continue in jobs unrelated to their degrees lose more 


earnings than workers who are not self-employed (Bender and Roche 2013). 


Job relatedness measures have been used to understand the dynamics of narrower labor markets 
or fields of study, including psychology (Rajecki and Borden 2009); economics and business (Robst, 
VanGilder, and Steinke 2016); and engineering technology (Frase, Latanision, and Pearson 2017; Kuehn 
et al. 2015). These analyses show that the prevalence and impact of working in an unrelated job varies 
by field of study—highlighted previously by Robst (2007a). However, as in much of the literature on job 
relatedness, none of these analyses explore how respondents’ understanding of what constitutes a 
“related job” varies by field. Respondents with differing backgrounds and experiences may understand 


the implications of the question differently. 


Although much of the existing research treats working in an unrelated job as an unexpected, costly 
experience, some workers with good job prospects outside of their field of study choose to work in 
unrelated jobs willingly and rationally. Although workers with high job relatedness consistently earn 


more than those with low job relatedness, workers who self-select unrelated jobs may nevertheless 
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earn more in their unrelated job options than they would have earned in an available related job. While 
unrelated jobs pay less on average, they may be a better deal for workers who self-select into them. An 
extreme and cliché example of this sort of self-selection would be a physics graduate working as a 
“quant” on Wall Street because it pays substantially more than working in a postdoctoral position or as a 
faculty member at a small liberal arts college. More typical cases might include science and engineering 
majors taking up unrelated management positions. Many researchers have recently explored self- 
selection into related and unrelated jobs (Gilmartin et al. 2018; Lee and Sabharwal 2016; Melguizo and 


Wolniak 2012; Zhu 2014).° 


The National Center for Science Engineering Statistics largely focuses on the US, but job 
relatedness is a fertile area of research in other countries too. In addition to cross-country studies 
(Morgado and colleagues 2014; Verhaest, Sellami, and van der Velden 2017), researchers have studied 
job relatedness in Belgium (Sellami and colleagues 2017); Canada (Boudarbat and Chernoff 2012); 
France (Béduwé and Giret 2011); Germany (Ortiz and Kucel 2008); Ireland (Kelly, O’Connell, and Smyth 
2010); Italy (lammarino and Marinelli 2015); Korea (Kim and Park 2016); Pakistan (Faroog 2011); Spain 
(Oritz and Kucel 2008); and Switzerland (Pecoraro 2014). Findings are comparable to those in the 
literature on the US: low levels of job relatedness are associated with reduced earnings and job 
satisfaction. The lacuna in that literature is also comparable to that of the US: job relatedness measures 


with some prima facie plausibility are analyzed, but without in-depth exploration or validation. 


The Task Approach to Labor Markets 


Job relatedness measures vary across workers because occupations are more fluid and less 
homogenous than standard occupational classifications imply. Recently, labor economists have 
considered jobs as bundles of tasks that can be performed using various skills. Tasks are allocated 
through normal processes of comparative advantage and are influenced by technological progress. 
Autor and Handel (2013) argue that analyzing job tasks instead of education levels is integral to labor 
economics because tasks differ from education in two major respects. First, tasks are not durable 
investment goods like education: Workers and employers can modify task inputs as job requirements 
change. Assigning workers into tasks implies no one-to-one mapping between a worker’s stock of 
human capital and the tasks performed. Second, tasks differ from traditional approaches to human 
capital because tasks are a high-dimensional bundle of activities; they must be performed jointly to 


produce output. 
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The task approach to understanding labor markets emerged from the inability of what Acemoglu 
and Autor (2011) call the “canonical model” to fully explain many labor market phenomena. The 
canonical model draws an equivalence between workers’ skills and their job tasks; technology is either a 
substitute for or acomplement for labor, and this relationship is fixed. Technologies cannot replace 
particular tasks in the canonical model because the assignment of tasks to labor is static (Acemoglu and 
Autor 2011). However, a large body of research shows that capital can simultaneously substitute for 
and complement an occupation’s narrower tasks, suggesting that analyzing labor markets at a task level 


is useful (Autor 2013). 


Autor, Levy, and Murnane (2003) provides an early empirical foundation for this approach. They 
test a “routinization hypothesis” whereby jobs with more routine task inputs are more likely to be 
automated (i.e., substituted for capital) than those with less routine tasks. The task approach to labor 
markets helps clarify which workers are vulnerable to trade flows (Antras, Garicano, and Rossi- 
Hansberg 2006; Blinder 2009; Grossman and Rossi-Hansberg 2008; Jensen and Kletzer 2010) and 
technological change (Autor and Dorn 2013; Autor, Levy, and Murnane 2003; Goos and Manning 2007; 
McKinsey Global Institute 2017). 


Empirically testing task models requires datasets that capture unique tasks performed on the job. 
The Occupational Information Network (O*NET), the most commonly used data on job tasks for the US, 
provides information on the knowledge, skills, and abilities required to work in many detailed 
occupations. We use the O*NET data in this report to supplement the NSCG. The major limitation of the 
O*NET data for job relatedness is that tasks are identified at the occupational level, not the worker 
level, preventing analysis of within-occupation heterogeneity. These data are also difficult to revise 


consistently and therefore only provide a static picture of job tasks and other characteristics. 


Autor and Handel (2013) collects new data as a part of the Princeton Data Improvement Initiative 
survey on the job activities across various domains for a representative sample of US workers. Their 


analysis found that person-level measures of job tasks have predictive power for earnings. 


A longitudinal dataset created by Stinebrickner, Stinebrickner, and Sullivan (2017), the Berea Panel 
Study, directly measures job tasks as reported by workers. Analysis of the Berea Panel Study identified 
a strong relationship between wages and job tasks, again validating the task model’s usefulness, even 
with data limitations. One study using German employment survey data found that changes in task 
composition of employment in between 1979 and 1999 are “primarily accounted for by changes in task 
inputs within detailed occupations rather than shifts in employment across occupational categories” 


(Autor 2013).* 
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Although the task approach tends not to focus on detailed occupations, some studies have analyzed 
the job tasks performed by science, technology, engineering, and mathematics (STEM) workers. 
Brunhaver and colleagues (2018) argues that STEM degree holders are especially prone to frustration 
with nontechnical professional skills and tasks on the job. STEM graduates perceive their degrees as 
highly technical and not requiring high-level professional skill, which influences their perception of their 
current job and can lead them to think less of these tasks when considering their job relatedness. Frase, 
Latanision, and Pearson (2017) surveyed employers of engineering technologists and engineering 
technicians to understand the task differences between those workers. The surveys suggested that 


technicians had more restricted task sets and were more likely to do maintenance and testing. 


Use of Knowledge on the Job 


The most obvious dimension along which workers may think about the relationship between their job 
and their degree is the level and type of knowledge used on the job. If knowledge requirements strongly 
align with a worker’s field of study, their job should be closely related. This section explores the strength 


of the relationship between knowledge requirements and job relatedness. 


Unfortunately, the NSCG does not ask respondents about the knowledge requirements of their job. 
The most comprehensive data on occupational knowledge requirements are available in the US 
Department of Labor’s O*NET database discussed previously. Knowledge requirements are recorded in 
O*NET by the importance of a domain of knowledge to a job and by the level of a domain of knowledge 
required for a job. Importance indicates how essential a knowledge or skill is, and level indicates the 
degree of mastery required. For example, engineering knowledge would be of high importance for both 
an engineer and an engineering technician, but the level of knowledge required is higher for an engineer. 
The O*NET database uses a one-to-five scale for importance measures and a zero-to-seven scale for 


level measures. For this report, we rescale both measures to a zero-to-ten scale. 


We merge the O*NET data onto NSCG respondents’ records using their occupational category to 
provide the best information on the importance and level of knowledge of their field of study required for 
their current job (figure 1). The distribution of knowledge importance and required knowledge levels for 
STEM majors in the NSCG for each job relatedness category is presented in figures 1 and 2, 
respectively. The importance and level of knowledge required for the job are represented on the 
horizontal axis using a scale from zero to ten (with zero indicating low knowledge requirements and ten 
indicating high knowledge requirements). Respondents who work in unrelated jobs have lower 


knowledge importance for their field of study than respondents in somewhat or closely related jobs. The 
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average importance of arespondent’s field of study is 3.3 for respondents in unrelated jobs, 5.7 for 


somewhat related jobs, and 7.8 for closely related jobs. 
FIGURE 1 


Importance of Knowledge of a STEM Major’s Field of Study for Their Occupation 
By job relatedness category 


mNotrelated @ Somewhat related @ Closely related 


Share of workers 
30% 


25% 
20% 
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5% 
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Importance of knowledge 


URBAN INSTITUTE 
Sources: 2013 and 2015 NSCG. Knowledge requirements are from the US Department of Labor’s O*NET database. 
Notes: Distributions are calculated for the analysis sample of STEM graduates in the 2013 and 2015 NSCG. Importance of 
knowledge of each respondent’s field of study to their occupation from the O*NET database is rescaled to a zero-to-ten scale and 
plotted separately for each job relatedness category. 


Asimilar pattern emerges for level requirements (figure 2). The average level of knowledge of a 
respondent’s field of study required for their job is 3.1 for unrelated, 5.3 for somewhat related, and 7.0 
for closely related jobs. The differences between these averages are all statistically significant. Despite 
these statistically significant differences in average knowledge requirements, all three job relatedness 
categories exhibit wide variability of knowledge requirements for their field of study within the job 
relatedness category. Approximately 18 percent of respondents in unrelated jobs still have higher 
knowledge importance and level scores in their field of study than the average for respondents in 
somewhat related jobs. This distribution suggests that knowledge requirements alone cannot 


definitively answer whether a job is related or not, and that within-occupation knowledge varies greatly. 
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More factors explain respondents’ assessment of job relatedness than just the use of knowledge from 
their major field of study. Other job characteristics influence a respondent’s interpretation of the NSCG 
question. We explore these other job characteristics using multivariate analysis of the NSCG data and 


interviews with employed college graduates. 
FIGURE 2 


Level of Knowledge of a STEM Major’s Field of Study Required for Their Occupation 
By job relatedness category 
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Sources: 2013 and 2015 NSCG. Knowledge requirements are from the US Department of Labor’s O*NET database. 
Notes: Distributions are calculated for the analysis sample of STEM graduates in the 2013 and 2015 NSCG. Knowledge levels of 
each respondent’s field of study required for their occupation from the O*NET database is rescaled to a zero to ten scale and 
plotted separately for each job relatedness category. 


Multivariate Analysis 


Job relatedness is typically used as an independent variable to predict pay differentials (Robst 2008, 
2007 a, 2007b) or job satisfaction (Bender and Heywood 2009). We use job relatedness as a dependent 
variable to understand the factors associated with identifying a job as related or unrelated. As noted 
above, the multivariate analyses distinguish between factors that cause a graduate to work in an 


unrelated job, such as family formation, lack of available work in a related job, or weak job-finding 
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networks, and factors that induce a graduate to identify a job as unrelated, such as job tasks, 
knowledge requirements, job tenure, or supervisory responsibilities. We are only concerned with the 
latter to better understand how workers think about the relationship between their highest degree and 


their job. 


Data 


Data for the multivariate analysis comes from two sources: the NSCG and the knowledge, skill, and 
ability requirement fields of the O*NET data.° The NSCG is a nationally representative survey of college 
graduates fielded every two or three years by the National Science Foundation. Since 2010, the NSCG 
has used the American Community Survey as a sampling frame. The survey collects demographic, labor 
market, and educational data that are standard in federal surveys. It also collects information on issues 
important to the National Science Foundation, including information on research funding, job tasks, and 
participation in professional associations. This multivariate analysis will model NSCG respondents’ job 


relatedness by various job characteristics using an ordered logit model. 


The first independent variable is the number of years since the respondent earned their highest 
degree. A consistent finding in the literature is that job relatedness declines with age as workers 
disperse into other jobs and attain new skills (Bauer 2002; Bender and Heywood 2011; Biddle and 
Roberts 1994). A respondent’s field of study (and therefore their job) may change dramatically over 
time, reducing how the job relates to the respondent’s degree. Financial support for research is another 
important source of validation for STEM workers. Including research support variables in the models 
can determine whether workers who engage in federally funded research and development are more 


likely to report being in a related job than those who do not. 


One of the most important factors determining a respondent’s job relatedness is the tasks they 
perform on the job, which are called “work activities” in the NSCG. The survey provides information on 
the tasks that respondents engage in on the job and those that consume most of the respondent’s time. 
To arbitrate between major and more incidental job tasks, the analysis includes dummy variables 
indicating whether a task is the respondent’s primary or secondary task (tertiary and other tasks are not 
included). The models also include measures of knowledge and skill importance, discussed previously. 
The importance scales are highly correlated with the level scales for almost all fields, so the multivariate 
analysis uses only the importance scales. Finally, the models include dummy variables that indicate 


whether the respondent attends professional associations in their field of study. 
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The models do not control for respondents’ occupations. Controlling for occupation would allow us 
to identify which occupations have higher or lower relatedness levels but would prohibit isolating which 


job characteristics are associated with low job relatedness. 


Table 1 shows descriptive statistics on the analysis variables for the full sample of employed STEM 
graduates in the 2013 and 2015 NSCG waves and descriptive statistics on graduates’ job relatedness 
responses. Many job characteristics and tasks, besides required knowledge of a respondent’s field of 


study, are associated with the job relatedness. 


TABLE 1 
Descriptive Statistics for Full Sample by Job Relatedness Category 
Somewhat Closely 

Variable Fullsample _ Not related related related 
Federal research and development support 15.7% 8.6% 13.7% 17.8% 
Individuals directly supervised 2.9 (10.5) 2.6 (9.3) 3.0 (9.2) 3.0 (11.0) 
Primary or secondary work activity is 
Accounting 9.3% 19.2% 14.1% 5.7% 
Basic research 6.0% 3.8% 4.3% 7.0% 
Applied research 11.6% 4.2% 9.8% 13.7% 
Development 8.9% 3.9% 9.8% 9.6% 
Design 9.2% 4.4% 9.8% 9.9% 
Computer applications 13.8% 78% 15.2% 14.5% 
Human resources 4.2% 6.5% 6.5% 3.0% 
Management 33.5% 32.7% 40.5% 31.2% 
Production, operations, maintenance 7.0% 9.6% 9.7% 5.5% 
Professional services 34.6% 13.0% 17.1% 44.9% 
Sales, purchasing, marketing 12.3% 35.5% 17.5% 5.6% 
Quality or productivity management 8.0% 8.4% 11.3% 6.8% 
Teaching 15.1% 7.3% 10.4% 18.2% 
Knowledge and skill requirements for job 
Knowledge of own field of study 8 (3.0) .3 (2.4) 7 (2.8) 7.8 (2.4) 
Administrative knowledge 4 (1.2) 6 (1.3) 6 (1.4) 5.3 (1.2) 
Clerical skills 4 (1.2) 9 (1.6) 5 (1.3) 4.3 (1.1) 
Customer relations skills 4 (1.5) 9 (1.2) .3 (1.4) 6.4 (1.5) 
Sales knowledge . (1.6) 8 (2.1) 2 (1.9) 2.6 (1.3) 
Attended professional meeting in past year 38.4% pane eae 44.2% 
Member of professional society 48.4% 27.1% 39.3% 55.7% 
Share of total (weighted) 100.0% 9.0% 22.7% 68.0% 
Observations (unweighted) 88,544 7,993 20,147 60,404 


Sources: 2013 and 2015 NSCG. Knowledge and skill requirements are from the US Department of Labor’s O*NET database. 
Notes: All statistics are reported in means. Standard deviations are in parentheses for continuous variables. All descriptive 
statistics are weighted to be representative of the national population of employed college graduates with their highest degree in 
a STEM field. 
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Research and development tasks, computer application tasks, and teaching have a clear positive 
correlation with job relatedness for STEM graduates. Professional service tasks are also strongly 
associated with job relatedness, reflecting the provision of many technical and scientific services 
through the professional services industry. NSCG respondents who report accounting, human 
resources, production, or sales as their primary or secondary work activity are less likely to report their 
jobs as closely related to their field of study. Management activities do not have a distinct positive or 
negative relationship with job relatedness. The number of people that a respondent directly supervises 
(a variable closely related to management tasks) also has no statistically significant descriptive 
association with job relatedness. Reflecting the results presented in figures 1 and 2, respondents who 
work in jobs where knowledge of their field of study is more important are more likely to consider their 


job closely related to their field of study. 


Methods 


We use an ordered probit model to estimate the relationship between job characteristics and job 
relatedness responses in the NSCG. The ordered probit accounts for the fact that the job relatedness 
variable is a discrete characterization of a continuous latent relatedness variable. Workers’ views of 
their job relatedness exist on a spectrum, which they map on to the three available survey responses. 


The dependent variable in the ordered probit model is the NSG job relatedness variable. 
The ordered probit model is motivated by assuming the existence of a latent variable: 
Yi = BX, + & 


Where Vi is an (unobserved) latent degree of job relatedness underlying discrete responses to the 
job relatedness question. The latent variable is a function of the set of controls X;, described in the data 
section. Observed levels of job relatedness are discrete and determined by the model as follows, 


assuming a normal distribution for the latent variable model: 


P(unrelated) = y(—-B'x;) 
y; = \P(somewhat related) = (u-— B'x;) — p(-f’x;) 
P(closely related) = 1-—o(u- B’x;) 


The parameters of the model are estimated using maximum likelihood including the cutoffs 
between the discrete job relatedness categories. Because many respondents appear in both the 2013 


and 2015 NSCG waves, we also estimate a random effects ordered probit model. 


HOW DO WORKERS THINK ABOUT THE RELATION OF THEIR JOB TO THEIR DEGREE? ees 


Results 


Table 2 presents marginal effects from the ordered probit models for all STEM graduates responding to 


the NSCG. The qualitative results are very similar between the models with and without random effects. 


Job relatedness increases, though at a decreasing rate, as respondents get further away from their 
highest degree. This is inconsistent with expectations and the descriptive statistics reported in table 1 
because of conditioning on other job characteristics in the models that vary with age, most notably task 


assignments. 


Federal research and development support is a strong predictor of job relatedness. Again, the effect 
of federal research and development support is estimated conditional on job tasks (including research 
and development tasks), so the coefficient is not simply proxying for participation in research activities. 
It indicates that, even conditional on doing research, receiving federal support increases the likelihood 


that a respondent will identify their job as related to their highest degree. 


The work activities with the strongest positive relationship with job relatedness are professional 
services and teaching. Typically, professionals and teachers are required, or at least expected, to work in 
their fields of study. Classic STEM work activities, including research and development, design, and 
computer applications, have a strong positive relationship with job relatedness. Some work activities 
indirectly related to STEM, such as management and production work, have a modest but positive 
relationship with job relatedness. Three task assignments—accounting, human resources, and sales 
tasks— were associated with a reduced likelihood that STEM graduates would identify their job as 


related to their highest degree. 


Though many job tasks are associated with higher levels of job relatedness, most knowledge and 
skills were not particularly important. Among the knowledge and skill requirements, only knowledge of 


arespondent’s own field of study clearly factored into assessments of job relatedness. 


Attending professional meetings and membership in a professional society have a fairly strong 
association with high job relatedness. The combined effect of membership in a professional society and 
attending meetings is comparable with doing applied research as a primary or secondary work activity. 
Sales knowledge and clerical skills were two occupational knowledge requirements associated with 
reduced job relatedness. The results for job tasks also showed negative job relatedness for sales and 


administrative support tasks. 
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TABLE 2 
Determinants of STEM Graduates’ Job Relatedness Responses 
Random effects 


Variable Ordered probit ordered probit 
“Yearssincehighestdegree 0005" Oo10* | 
Years since highest degree, squared -0.000*** -0.000*** 
Federal research and development support 0.184*** 0.297*** 
Individuals directly supervised 0.003*** 0.004*** 
Primary or secondary work activity is 
Accounting -0.119*** -0.226*** 
Basic research 0.356*** 0.697*** 
Applied research 0.422*** 0.701*** 
Development 0.386*** 0.496*** 
Design 0.299*** 0.502*** 
Computer applications 0.400*** 0.536*** 
Human resources 0.022 -0.067 
Management 0.199*** 0.247*** 
Production, operations, maintenance 0.105*** 0.124*** 
Professional services 0.786*** 0.963*** 
Sales, purchasing, marketing -0.220*** -0.367*** 
Quality or productivity management 0.192*** 0.138*** 
Teaching 0.561*** 0.823*** 
Knowledge and skill requirements for job 
Knowledge of own field of study 0.189*** 0.294*** 
Administrative knowledge 0.081*** 0.117*** 
Clerical skills -0.048*** -0.032*** 
Customer relations skills 0.010** 0.018** 
Sales knowledge -0.046*** -0.125*** 
Attended professional meeting in past year 0.217*** 0.301*** 
Member of professional society 0.253*** 0.412*** 
Observations 88,544 88,544 
Cutoff between unrelated and somewhat related 0.749 0.571 
Cutoff between somewhat and closely related 1.833 2.630 
Rho -- 0.687 
Pseudo R-squared 0.239 -- 


Sources: 2013 and 2015 NSCG. Knowledge and skill requirements are from the US Department of Labor’s O*NET database. 
Note: Year fixed effect not reported. 


Perspectives from Employee 
Interviews 


We contacted human resources and public relations departments at varying STEM employers to 
request referrals of employees that might be interested in participating in voluntary interviews. The 
employers contacted included National Laboratories, companies affiliated with national science and 


engineering associations, urban planning departments, health care systems, defense contractors, and 
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technology and social media companies. Most employers were not willing or able to provide 
interviewees, but we interviewed fifteen college graduates employed in varying occupations.° Although 
the interviewees were not required to have STEM degrees, most had STEM backgrounds, both because 
of the types of employers we contacted and because our initial invitation to participate indicated that 
the research effort was funded by the National Science Foundation and focused on STEM graduates. 
The interviews were semistructured; though we used a standard interview protocol, many questions 
were open ended, resulting in varied content. The interview guide is provided in appendix A. The 
interviews began by replicating the NSCG survey question on job relatedness, allowing respondents to 
identify their jobs as closely, somewhat, or unrelated to their fields of study. We then asked 
interviewees what factors of their jobs and their education they considered when answering the job 


relatedness question. 


Table 3 provides a summary of the fifteen interviewees. Many were senior managers, scientists, or 
engineers, with educational attainments ranging from bachelor’s degrees to doctorates. Almost half of 
the interviewees had degrees in engineering; others held degrees in life and natural sciences. Because a 
STEM degree was not a requirement, four respondents had non-STEM degrees (two in urban planning, 
which included some engineering content). More of the interviewees held doctorates than the general 
public because of the strong representation of scientists and engineers. Interviewees were more likely 
to indicate their job was closely related to their highest degree than would be expected from the NSCG 
data. This could be attributable to self-selection; individuals that felt a strong connection between their 
jobs and highest degree might have been more interested in participating in an interview about job 
relatedness. Respondents may also have answered in a way that painted themselves in a positive light, 


fearing a negative response from the interviewer to a not related answer. 


Several major themes emerged as respondents explained the reasoning behind their job 
relatedness answers. The two most important determinants of job relatedness were the respondents’ 
understanding of their degree content as narrow or broad and their most important job tasks. Among job 
tasks, management responsibilities were discussed most consistently across all interviewees, although 
they disagreed on whether management responsibilities made their jobs less related to their field of 
study. Many interviewees discussed the importance of on-the-job learning for filling in the gaps 
between the knowledge taught in their degree programs and the knowledge required for their job. 
Finally, the interviews highlighted the role of professional associations, certifications, and expectations 
in assessing job relatedness. These factors were generally less important than job tasks and the breadth 


of the interviewees’ degrees. 
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TABLE 3 
Overview of Interviewees 


Subject 


number Highest degree 


Biomedical engineering, PhD 


1 

2 Chemistry, MS 

3 Nuclear engineering, PhD 
4 Law and business, BA 


5 Urban planning, MA 

6 Computer science, BS 

7 Mechanical engineering, PhD 

8 Aerospace engineering, MS 

9 History, BA 

10 Nuclear engineering, MS 

11 Civil engineering, MS 

12 Urban and regional planning, 
MA 

13 Metallurgical engineering, 
PhD 

14 Geochemistry, PhD 

15 Pharmacology and molecular 


sciences, PhD 


Source: Authors’ interviews. 


Current occupation 
Senior staff scientist, biosciences 
Scientific associate 
Staff scientist, radiation protection 
Program manager, corporate social 
responsibility 
City planner 
Software engineer 
Senior manager in structural mechanics 
Senior manager 
Senior program manager, corporate 
citizenship 
Program manager in research and 
development 
Senior manager in planning department 
Wireless planner in planning 
department 
Senior fellow in metallurgy and 
engineering 
Senior scientist, trace metal research 
Chief operating officer for bioenergy 
innovations and resource business 
development manager 


Job relatedness 
response 
Closely related 
Not related 
Closely related 
Somewhat related 


Closely related 
Closely related 
Closely related 
Closely related 
Not related 


Closely related 


Closely related 
Closely related 


Closely related 


Closely related 
Somewhat related 


Narrow Versus Broad Conception of Degree 


How broadly or narrowly a respondent construed their field of study shaped their sense of job 
relatedness. Some interviewees conceived their highest degree’s content as being broader than just the 
specific research questions that they personally pursued. A senior staff scientist with a doctorate in 
biomedical engineering suggested that “topically, not a lot of [her] work now currently involves 
biomedical research per se,” but “the process of science that [she] learned during [her] PhD prepared 
[her] for other research.” A program manager with a background in computer programming anda 
bachelor’s degree in law and business cited a unifying theme outside those fields, suggesting that “the 
one thing that underpins both [his] career choice and [his] choice of degree is that [his] favorite subject 
was math.” He described the law as being structured and logical, like coding and math, and emphasized 


the similar problem solving in his education and job. 


Sometimes the content of an interviewee’s degree was broader than the degree title implied. A 


senior fellow with a doctorate in metallurgical engineering noted that his 1960s-era education would 
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take a different name now. He described his degree as “not exclusive to metals because all materials are 
of interest, so a bit more general. All materials that have common characteristics and their use is 
something that, today, is more like material science.” Often interviewees identified components of their 
degree that were outside of the field of study. One subject noted that the strong emphasis on formal 
presentation skills in his engineering program helped him tremendously on the job. A senior manager in 
acity planning agency with a master’s degree in civil engineering described her degree as including 


broad skills like “drive, the desire to figure things out, and problem solving.” 


For asenior staff scientist with a doctorate in biomedical engineering, the broad, interdisciplinary 
nature of his field of study meant that he could work across different sciences. He noted, “I worked ina 
very interdisciplinary field. If | looked at the CV of myself and most of my colleagues, the degree names 
would be all over the place: mechanical engineering, chemical engineering, materials science, chemistry. 
Everyone’s doing the same topics, and we speak slightly different languages.” For him, “having an 
interdisciplinary degree helps [him] translate. So in that way, the degree provides the sort of Rosetta 
Stone that translates between the two disciplines.” This respondent indicated his job is closely related 


to his degree, despite many other fields of study being part of his daily work. 


These broad conceptualizations of a degree program’s content were typically associated with high 
job relatedness. A city planner with a master’s degree in urban planning highlighted the unknown 
elements of the job for which her degree’s broad content prepared her, mentioning that “education gave 


[her] tools to address new problems, but not a sense of what those problems would be.” 


Having an interdisciplinary degree helps me translate. So in that way, the degree provides the 
sort of Rosetta Stone that translates between the two disciplines. 


—Senior staff scientist with a doctorate in biomedical engineering 


Other interviewees understood their highest degree as being much narrower. A scientific associate 
with a master’s degree in chemistry considered her degree in terms of her own subfields, saying, “| 
studied inorganic chemistry when | was in graduate school, and I’m not doing that at all anymore... The 
industry for inorganic chemistry is more in paints and coating. The instruments that | work on, we look 
at polymers and biological systems. So, if | had studied polymer science or chemistry that would be close 


to what I’m doing.” Not all respondents that described their degree narrowly said that their jobs were 
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unrelated to their degree; some narrowly construed degrees were still related. A program manager at a 
National Laboratory with a master’s degree in nuclear engineering described his studies as 
“radiochemistry and radiochemical processing” of a particular isotope. He felt that his work closely 
related to his degree because he used the processing techniques he learned in school, despite working 
with a different isotope. Much like the scientific associate who studied inorganic chemistry, he 
considered his field of study as more narrow. Unlike the staff associate, he still thought of his job as 


related. 


Both broad and narrow understandings of the subject’s highest degree are coherent and 
reasonable. Nevertheless, subjects’ interpretation of the question and their degrees varies how the job 
relatedness question is answered. Interviewees with doctorates’ conceptions of unrelated work may 
differ from those with other educational levels because of the highly specialized nature of their 


research. 


Job Tasks 


When first asked, interviewees tended to base their job relatedness assessment on titles rather than the 
content of their jobs. But when probed on particular tasks, respondents often described management 
and other nontechnical tasks as important in their day-to-day work. This response pattern corroborates 
the experience in interviews with engineering graduates conducted by Brunhaver and colleagues 
(2018), which found that respondents initially minimized the role nontechnical tasks played in their 
work until they were asked how their views of an engineering job changed over time. One interviewee, a 
wireless planner with a master’s degree in urban and regional planning, answered quickly that her job is 
closely related to her degree. She described her technical job functions as reviewing and permitting 
installation for wireless telecommunications facilities. When probed on particular tasks, she 
emphasized communication as a crucial skill, despite her job’s technical nature. This evolving response 
calls into question the NSCG survey question’s ability to accurately capture the fluid nature in which 


workers perceive job relatedness. 


Research and development was an important job task discussed by many interviewees. A chief 
operating officer with a doctorate in pharmacology and molecular sciences described his job as 
somewhat related to his degree partly because he had not conducted any bench science since earning 
his degree. His departure from research was planned: “I knew that | was not going to continue with 
bench research or follow a teaching pathway, so | guess | didn’t have expectations, but | knew that what 


| wanted to do with my degree was get into roles where | could help PhDs to do research. The career 
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path I’ve taken has allowed me to do that.” This subject emphasized that his job was still somewhat 
related because he uses his doctorate “extensively” to complete job tasks, including proposals and 
business development. The doctorate helped him relate with researchers and “understand what it takes 


to get the research done, published, and get it funded.” 


Some tasks were perceived as conflicting with the ideals or objectivity of science. A staff scientist 
with a doctorate in nuclear engineering mentioned that his research is “intended to serve society,” and 
that “many results are directly ingested by federal agencies to provide a foundation for regulations.” He 
cautioned that “there’s a line you have to draw as a scientist. If [he thinks] about cost implications, legal 
impacts, etc., then [he’s] being unfaithful to the science side of things. There’s some grey areas there, 


but, intentionally, a scientist is supposed to stay away from focusing on the broader picture.” 


Management 


Management and supervisory tasks were important mediators of respondents’ assessments of job 
relatedness, though their significance for job relatedness varied. Several respondents spent 
considerable time in management tasks. A program manager with a master’s degree in nuclear 
engineering mentioned that the extent of her management activities surprised her. A senior manager at 
a planning department with a master’s degree in civil engineering noted that each week she spends “at 
least 60% of [her] time in meetings that are not engineering related.” Despite her substantial 
management responsibilities, the senior manager still identified her job as closely related to her highest 
degree. A senior fellow with a doctorate in metallurgical engineering corroborated the time-consuming 
nature of management: “Everything | do personally is involved with materials, but management 
functions simply added another dimension in terms of time commitments.” A software engineer who 
reported his job as closely related to his computer science degree agreed that management tasks could 


be time consuming. But, the technical skills he learned in school were critical for management tasks. 


Asenior program manager with a bachelor’s degree in history also agreed on the time-consuming 
nature of management. She had previously worked in a position with greater managerial commitments 
before moving to a new position with fewer commitments. Reflecting on the difference between the 
two jobs, she suggested that “the only real difference between when [she] had more management 
responsibilities and now is that [she] had less time [when she had more management responsibilities].” A 
senior scientist with a geochemistry doctorate reported that there were “always things in management 


that we have to respond to. In this day and age of emails, that eats up time.” Not all subjects were tied up 
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in management. A chief operating officer with a doctorate in pharmacology and molecular sciences 


described spending only about 5 percent of his time supervising three employees. 


Many of the people that | manage hold degrees and have research backgrounds similar to my 
own. Managing them therefore requires knowledge of the same areas. 


—A senior manager with a doctorate in structural mechanics 


Many subjects recounted increased time spent on management tasks as they moved up the ranks in 
their company, though how this trend affected their job relatedness varied across respondents. A staff 
scientist with a doctorate in nuclear engineering described her nonresearch tasks as “just part of being a 
scientist” and not taking up much time. A senior manager with a doctorate in structural mechanics who 
described a similar trajectory of increased management activities explained, “Many of the people that | 
manage hold degrees and have research backgrounds similar to my own. Managing them therefore 
requires knowledge of the same areas. If you’re asking whether | solve equations every day, | don’t do 


that anymore, but | use what | know about mechanical engineering.” 


Others saw management differently. A senior staff scientist with a doctorate in biomedical 
engineering indicated that “[t]he job that [his] degree doesn’t prepare for is managing people. You do 
more and more managing of people and less and less of the science itself.” The senior fellow with a 
doctorate in metallurgical engineering agreed that management was not related, noting that, “short of 


the management work [he has] had, all the technical work [he has] done relates back to [his] training.” 


On-the-Job Learning 


Several interviewees cited on-the-job learning as an important intermediator between their highest 
degree and their job. The interview guide did not include any questions about on-the-job learning, but 
interviewees independently raised the issue. Formal and informal learning on the job was typically 
characterized as a bridge between the knowledge attained in the classroom and the skills necessary for 
the job. A senior manager with a doctorate in mechanical engineering emphasized that “the main thing is 
that there is more to education than you learn in the books. On the job, you learn a lot of important 


things that help you apply your school work but that you’d never learn in school.” A wireless planner 
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with a master’s in urban and regional planning described her transition from transportation to 
telecommunications planning as mediated by a job in consulting for a telecommunications company, 
which “gave [her] exposure to various agencies [in the area] and helped deepen [her] knowledge about 
planning in general.” This on-the-job learning experience mediated her assessment of her job as closely 
related to her degree. A staff scientist with a doctorate in nuclear engineering noted, “I feel like | was 
trained as much as | could have been, but the field is so complicated that school doesn’t prepare you. 


[You] have to learn as you go.” 


The formality of on-the-job learning varied across work environments. A program manager at a 
National Laboratory with a master’s in nuclear engineering indicated that he picked up his cost 
estimation skills informally. He noted, “Cost estimating | did not learn from school, that just comes from 
balancing your checkbook and doing your tax returns.” In contrast, a program manager for corporate 
responsibility with a substantial technology portfolio learned her technical skills in nondegree formal 
trainings and independent study. Others cited on-the-job learning as both formal and informal. A 
program manager with a bachelor’s in law and business noted that a previous employer, a large 
management consultancy, “put us through a three-month intensive training course. Depending on the 
project, there was potentially more learning on the job.” This interviewee reported that her company 
was intentionally recruiting program managers with nontechnical degrees who could be taught 


technical skills on the job to ensure a breadth of expertise. 


On the job, you learn a lot of important things that help you apply your school work but that 
you'd never learn in school. 


—A senior manager with a doctorate in mechanical engineering 


Developments in the field and in technology and techniques sometimes led to on-the-job training. A 
senior scientist who thought of his job as closely related to his doctorate in geochemistry stated, “[I] had 
to learn new equipment. The basic equipment | still have in my lab, but | use it less because there’s new 
techniques. In a job function, at the end of the day, the projects have to move, so | had to learn new 


techniques.” 
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Professional Organizations and Credentials 


Several interview respondents indicated either that they had some professional certification or license 
(e.g., a professional engineering [PE] license) or were a member of a professional organization. These 
certifications and memberships can validate or cultivate occupational identities that could inform job 
relatedness assessments. Most respondents with certifications, licenses, or association memberships 
agreed that those contributed to their occupational identity. A senior scientist with a doctorate in 
geochemistry said, “[Professional associations] play an intricate role; they guide new ideas, things | 
should be interested in. They help shape that because you’re learning from other people, and they 
influence what you find interesting. [It can] drive what | want to do in the lab.” Others were more 
skeptical; a staff scientist with a doctorate in nuclear engineering suggested that his professional 


association was “a bunch of PhDs running around patting themselves on the back.” 


Astaff scientist with a doctorate in nuclear engineering noted the importance of social relationships 
outside professional associations, saying they are a factor in the assessment of one’s job relatedness: 
“TThe] social connections | made during graduate school are the foundation of [my] professional 


network right now. My mentor and his collaborators [from graduate school] are now my bosses.” 


For most engineers, a PE license was not considered particularly important, although some had one. 
This is because PE licenses are most common in civil engineering, and our interviews drew more heavily 
ona research and development workforce (table 3). A senior manager ina city planning agency with a 
master’s degree in civil engineering stressed the importance of the PE for her job. Because she is the 
only employee in the agency with a PE license, she noted that other teams seek out her input or review 
on engineering related projects. The PE license clearly identifies her as an engineer and shapes her 
tasks. The same senior manager has a nondegree certificate in urban planning and noted its importance 


for her current job. 


Expectations 


Expectations anchor respondents’ assessment of their job. Everyone has a vision for what life as a “real” 
scientist or engineer entails. Frequently, these expectations are not met as graduates confront 
management, accounting, and other administrative elements of their jobs. All interviewees were asked 
whether the reality of their jobs conformed or clashed with their job expectations as students, but for 
the most part, prior expectations did not factor heavily in their job relatedness response. Several 


interviewees indicated that their education was so long ago, they could not remember what their 
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expectations were. Others said they were so focused on school that they didn’t have any strong 
expectations about what their career paths would look like. A scientific associate with a master’s degree 
in chemistry did affirmatively recall that she “expected at this point [she] would probably be in 


management.” 


Discussion and Recommendations 


Several common themes emerged from the NSCG data analysis and the employee interviews: required 
knowledge of a worker’s field of study mattered to a worker’s assessment of job relatedness, in both the 
NSCG and the interviews. However, knowledge requirements were by no means the sole determinant 
of job relatedness. Federal support for research, the time since a worker earned his or her degree, and 
participation in professional societies were also significant factors. Perhaps the most important 
determinant of job relatedness, though, was a worker’s job tasks. Classic STEM job tasks like research, 
development, and computer applications were of course positively associated with job relatedness. 
Management responsibilities were associated with higher reported job relatedness in the NSCG 
analysis than expected. The interviewees were more divided on whether management responsibilities 
positively contributed to job relatedness. Subjects whose management work required significant 
technical competence to be effective were more likely to report their job as closely related to their field 
of study. One-the-job training, which bridges the gap between formal education and work, emerged as a 
consistent and unexpected finding. On-the-job training in the knowledge and skills not provided in 


school helped keep the interviewees’ degrees relevant. 


The data analysis and interviews suggest several recommendations for future NSCG waves, 
including adjustment of question ordering, collection of more detailed task data, reinstatement of older 
job expectation questions, and enhancement of existing on-the-job training questions. Although careful 
planning and many competing priorities dictate the final content of the NSCG questionnaire, the 
National Center for Science Engineering Statistics should consider these possibilities for future waves 


of the survey: 


= Adjust job relatedness question ordering. Question ordering is a significant factor in 
understanding the NSCG job relatedness responses. The NSCG job relatedness question is 
asked early in the survey’s section on the respondent’s principal job, before questions on job 
tasks, supervisory responsibilities, job satisfaction, federal funding, certifications, and other 


work-related experiences. The literature on survey design suggests that more specific and 


(24 HOW DO WORKERS THINK ABOUT THE RELATION OF THEIR JOB TO THEIR DEGREE? 


straightforward questions are best placed at the beginning of the survey, and more difficult, 
general, or intrusive questions should be asked later, cuing respondents that the more general 
questions should be considered separately from the specific questions (Newcomer and Triplett 
2015). Earlier questions can also help define certain terms or concepts and provide context that 
respondents can draw upon in later questions (Groves 2004). An alternative ordering where job 
relatedness is asked at the end of the principal job section of the NSCG could help ensure 
responses better reflect respondents’ job characteristics. The job relatedness question is more 
encompassing than every other question in the principal job section (except for the job 


satisfaction questions) and may therefore be more meaningful if asked at the end of the section. 


= Revive job expectations questions. Interviews showed that respondents had varying 
expectations about their future careers when they earned their degree. Some fully anticipated 
their actual career trajectory, but others did not. The interviewees did not discuss their 
expectations as areason for considering their job related to their degree, but it informed how 
they framed their job history. It may be helpful to bring back job expectations questions from 
older versions of the NSCG. Box 2 shows a question about graduates’ expectations in the NSCG 


surveys conducted in and before 1999. 


= Collect more detailed task data. Task variation within occupation presents another 
opportunity to better understand and improve the NSCG question. A large and growing body of 
research using the O*NET database exposes task variation within occupations, complicating 
how researchers can analyze the relation between degree and occupation. A ranked scale for 
the importance or presence of tasks in a given occupation may better capture true job 


relatedness. 


= Enhance existing on-the-job training questions. Interviews proved that on-the-job training is 
an important bridge between school and work. Some training questions are already included in 
the NSCG, but these ask about intensity or duration of training, and subject matter of training 
would be useful. Accounting for informal educational experiences during formal schooling may 
further improve the accuracy of the NSCG question. Some interviewees noted the importance 


of informal educational pieces, like social networks, when assessing their job relatedness. 
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BOX 2 
Graduates’ Expectations Question in Older NSCG 


Thinking back to when you completed your highest degree, would you say your work during a TYPICAL 


week on this principal job is: 
[Mark one answer] 
= Very similar to what you expected to be doing 
= Somewhat similar to what you expected to be doing 


= Not very similar to what you expected to be doing 


Note: See the 1999 NSCG questionnaire at https://www.nsf.gov/statistics/srvygrads/surveys/srvygrads_1999.pdf 
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Appendix. Interview Guide 


Introduction and Confidentiality Statement 


Thank you so much for taking the time to speak with me today. My name is Daniel Kuehn, and| ama 
researcher at the Urban Institute, a nonprofit research organization based in Washington, DC. | am 
conducting a study, supported by the National Science Foundation, to understand how science and 
engineering graduates understand the relationship between their job and their field of study. The 
purpose of these interviews is to gain insights into how you think about the connection between your 


degree and your job. 


The study also involves the analysis of a large survey previously conducted by the National Science 
Foundation. My interview with you today will provide important context for the analysis of the survey 
data. There are no right or wrong answers; | am only interested in your own reflections on your job and 


your education. 


Before we get started with the questions, please be informed that your participation in this 
interview is strictly voluntary; you can choose not to answer any question, and you are free to leave the 
interview at any time. Although | will be taking notes during the interview, the information you provide 
will never be repeated with your name in any reports or in any discussions with anyone outside of 
Urban Institute, including your employer. The interview notes will be made available to other 


researchers, but no identifying or sensitive information will be released. 


Additionally, it would be very helpful if | could record the conversation so | don’t miss anything 
important in my notes. The recording will be deleted as soon as | have made a complete set of notes 


from the interview. 


Do | have your permission to record? Do you have any questions before we begin? 


APPENDIX (27) 


Interview Questions 


1. I'd like to begin with some basic background information on your education and your current 


job. What was the title of the principal job you held during the last week? 
2. What was the level of your highest degree?: 
a. Bachelor’s degree (e.g., BS, BA, AB) 
b. Master's degree (e.g., MS, MA, MBA) 
c. Doctorate (e.g., PhD, DSc, EdD) 
d. Other professional degree (e.g., JD, LLB, MD, DDS, DVM) 
3. What was your field of study for that degree? 
4. Towhat extent was your work on your principal job related to your highest degree? Was it... 
a. Closely related 
b. Somewhat related 
c. Notrelated 


5. Canyou tell us about how you think about your job’s relatedness? What factors do you weigh in 


making that judgement? 
a. Was this an easy or a difficult determination to make? 


6. What are your major job tasks? What role did these tasks play in determining the relation of 


your job to your field of study? 


a. [Probe: How does the interview subject interpret the relatedness of management, sales, or 


other tasks not obviously related to the academic content of the subject’s degree?] 


7. How did the reality of your job coincide or clash with your job expectations while you were 


earning your degree? 


a. [Probe: What job tasks surprised you? What tasks did you anticipate that did not end up 
being a part of your job?] 


8. Has your sense of the relation of your job to your field of study changed over time? If so, how? 


Was this because your job tasks changed over time, or for some other reason? 


9. Doyou feel that everyone holding your job title is [closely related/somewhat related/not 


related] to your field of study? 


Ea APPENDIX 


Notes 


1“Table 5. Employment in STEM Occupations by Field of Degree for the First Listed Bachelor’s Degree: 2012,” US 
Census Bureau Employment Statistics of College Graduates, accessed October 22, 2018, 
https://www.census.gov/data/tables/2012/demo/industry-occupation/college-graduates.html/. 


2 Keith A. Bender, and Kristin Roche, “Educational mismatch and the earnings distribution,” Southern Economic 
Journal (forthcoming). 


3 Daniel Kuehn, “Three essays on connecting to work,” (doctoral dissertation, American University, 2016). 


4 The data are from the Federal Institute for Vocational Education and Training (BIBB) and the Institute for 
Employment Research (IAB). |AB/BIBB employment data are widely used for studying the German economy and 
are particularly well suited to analyses of job tasks. 


5 We considered using other National Center for Science Engineering Statistics datasets, including the Survey of 
Doctoral Recipients and the National Survey of Recent College Graduates, but we ultimately rejected them 
because the NSCG allows for more detailed occupational codes and additional variables on job characteristics 
that the Survey of Doctoral Recipients and National Survey of Recent College Graduates do not include. Because 
the O*NET data is only available at the occupational level, we needed to use the most detailed occupations 
possible. 


6 The National Laboratories, urban planning departments, technology companies, and engineering services 
companies were the most responsive to our request for interviewees. We were unable to obtain interviewees 
from any defense contractors, health care systems, or social media companies. 
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